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ABSTRACT 

The final 1970 report of a 3-part study on regional 
education services focuses on a unit for (1) planning cooperative 
educational programs in terms of financial considerations, resource 
allocations, program operation, governance, and roles relative to 
operation; (2) making provisions for evaluation of the regional 
cooperative; (3) providing for change and innovation in the 
cooperative; and (4) evaluating the unit using a 4-phase evaluation 
design, A 10-item bibliography, 8 figures, and 1 appendix are 
included. (MJB) 
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BASIC PLANNING AND EVALUATION MODEL FOR COOPERATION IN 
PROVIDING REGIONAL EDUCATION SERVICES 

Education in the United States traditionally has been a State function 
with governance of education controlled by local boards of education. 

Each school unit was a self-contained entity providing the full range of 
services that it could afford or that its clientele required. In recent 
years, demands for more services from local schools are generating new 
concepts in organization for education. Many of these concepts are predi- 
cated upon notions of regional cooperation rather than totally local 
autonomy. 

An early thrust for cooperative activity in education came from the 
school study council movement started in the late 1940 *s by Dr. Paul 
Mort at Teacher’s College, Columbia University. A wide variety of coopera- 
tive activities has been developed in education since that time; Title III 
of the Elenentary and Secondary Education Act of 1965 (ESEA) provided more 
incentive for school districts to cooperate to improve education. Various 
intermediate units, boards of cooperative services, and structures for 
shared services have stressed cooperation and expanded services in educa- 
tion. Recently, the development of educational cooperatives has been viewed 
as one means to improve. education in Appalachia. This present model has been 
developed from an array of concepts for cooperation found in the literature. 

Introduction 

An initial assumption of this model is that the basic idea of coopera- 
tive action in education is sound and generally accepted; therefore, a long 
rationale does not need to be developed as part of this document. The 
general concept or framework of an educational cooperative and a basic 





3 



- 1 - 



definition are included in the following material quoted from The Educa- 



tional Cooperative ; Rationale Administration , Implementation . ^ (Although 

the definition pertains specifically to cooperatives in Appalachia, it is 

broad enough “to relate generally to cooperative action in education.) 

This program addresses itself to the problems related to the 
locus of change -- small schools and school districts, inadequate 
financial and huinau resources, and distance-time factors -- and 
attempts to provide a structure and process by which the in- 
ventions of change can make effective long-term contributions 
to the education of children and youth. 

The cooperative is an aggregation of people, ideas, 
money, and potential. Its organization is a confed- 
eration of local school districts which in concert with 
a state department of education and a local college 
or university voluntarily bind themselves :ogether 
to increase their capacity through a joint effort ... 

The establishment of the cooperative is based on the assumption 
that 1) -the ratio of needs to resources will likely remain un- 
favorable; 2) educational need will continue to be greater in 
Appalachia than in the nation as a whole; 3) local school districts 
will continue to serve as the legal agencies for conducting 
public education Jo. 4) any plan for improving educa- 
tional accessibility must deal effectively with an educational 
base structure which is essentially a closed system functioning 
with inadequate resources. . . . 

In summary, the concept of Educational Cooperative holds that 
education needs new directions and that the design for the new 
directions must begin with its basic structure. Further, it is 
assumed that only through a dramatic rearrangement of relation- 
ships among school districts, administrators, classroom teachers, 
boards of education, and lay citizens can a change, in structure 
be achieved. 

An Intermediate Unit to foster and encourage regional cooperative 
action in education is a form of educational cooperative and contains 
basic elements of cooperation; one of the most important being that 
participants will voluntarily relinquish some decision-making prerogatives 



i The Educational Cooperative ; Rationale , Administration , Implementation . 
Appalachia Educational Laboratory, Inc., Charleston, West Virginia, 

June, 1969. Preface to the Draft Copy. 
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in exchange for assistance in obtaining certain goals and in providing 
educational services that each member separately could not realistically 
expect to provide alone. Some forces encouraging cooperative action in 
education are shown in figure 1. 

The nomenclature ’’Unit” used throughout this paper to represent a 
cooperative model is intended to be a generic, and not a specific, term. 
Concepts underlying the general ’’Unit” include the concepts underlying 
intermediate units, or educational cooperatives, or boards of cooperative 
services, or other cooperative arrangements between and among school dis- 
tricts. As a prototype evolves to accomodate specific requirements of 
districts being served and of state-wide legal jurisdictions within which 
the Unit is being developed, an attempt must be made to provide an umbrella 
under which diverse organizational arrangements might fit. It is particularly 
important that the model be open-ended so that it can (1) create impetus 
for change; (2) develop new ways of serving education -- new systems, 
structures, procedures and elements directed at providing new ways of 
doing educational things better; (3) adapt to changes within it; (4) adapt 
to changes encouraged by systems surrounding it or impinging upon it; 

(5) expand its sphere of influence effectively; and (6) maintain its own 
operations through efficient efforts. 

Cooperative arrangements are more than the simple total of the resource 
allocations of each basic component of the Unit. The educational cooperative^ 
a multi-district confederation, provides the conceptual and organizational 

framework for local school systems to increase their capabilities to 

2 

produce quality education. 



^Ibid .. p. 3 
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Relationships within each cooperative endeavor in education may be 
unique. In some cases, uniqueness is applied by legislative mandate; in 
other cases by demographic or geographic factors; in other cases, by 
still other factors or forces. A basic planning and evaluation model 
for the Unit should consider as many variables as possible to provide 
maximum flexibility resulting from diverse needs of each cooperative 
endeavor. 

Planning 

Some basic goals for the Unit should be described in general terms. 

The following are logical general goals: 

1. To provide expanded and improved administrative organization 
for the service area 

2. To provide services that a single district would not easily be 
able to afford by itself 

3. To provide for the organizational and program maintenance 
necessary for the Unit to sustain itself 

4. To encourage and facilitate change and innovation through a 
variety of means 

5. To allocate a percentage of its resources for *’ resource producing" 
or "resource creating" activities and for planned high risk 
activities. 

6. To provide solutions through cooperative action for educational 
problems that may be difficult to alleviate without cooperation 

7. To provide the impetus for developing new systems, ideas, 
procedures and linkages for education 

The general objectives, and the specific enabling objectives established 
to operationalize the general objectives will play an important role 
both in the planning and the evaluation stages of the Unit. Without some 
unique goals to provide more than simple administrative improvement and 
shared services, there would seem little real reason for establishing yet 

-5- 
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another stratim of decision-making within the educational structure. 



A basic planning and evalution model should be developed in such a 
way that it (1) is functional for immediate implementation, (2) is 
theoretical and conceptually sound for developmental and/or resource- 
producing activities, (3) is projective to provide direction for loiig- 
range goals, and (4) contains provisions for economic efficiency as well 
as program effectiveness. This basic model might be described as a plan- 
ning model including suggestions for an evaluation component and comprised 



of several operationally definitive phases or levels of activities. Figure 
2 portrays a summary of major elements of each level of program development 
for a basic model which includes: 

1. basic elements necessary for initial organization and minimal 
operation of an educational cooperative unit. 

2. Some first-level program developmental ideas consisting of 
administrative and/or organizational adjustments 

3. second-level program developmental ideas including program 
and curricula modifications, as opposed to administrative 
modifications in level 1 

4. specific long-range plans and projections for educational innova- 
tion from baseline data developed out of levels 2 and 3 

Some representative program, components which the Unit should include, 
or for which plans should be made for future consideration are: 

1. special administrative services, such as computer-assisted 
administration 

2. developmental activity such as field services and assistance in 
location and/or allocation of resources 

3. continuing education for vocation and avocation ( new and experi- 
mental programs) 

4. long-range planning capability 

5. psychological services 

6. materials and media development 



PROGRAM AND SYSTEM DEVELOPMENT 
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Figure 2 . FIck -7 chart indicating emphasis of each level of program development 




7. special education ( where there is insufficient enrollment in 
individual districts for adequate programs) 

8. coordination of curriculum resources 

9. provision for inservice activities 

10. basic provisions for data gathering and research 

11. capabilities for revision and updating of baseline data 

12. communication and coordination with the State Education Agency 

13. programs relating to improved and expanded educational opportunities 

a. adult basic education 

b. adult education 

c. occupational education 

d. cooperative programs 

e. work-study programs 

14. assessment and evaluation, services 

15. provision for continuous self-evaluation and self-renewal allowing 
adaptation of the total organizational environment to include or 
exclude discrete elements of the new organizational arrangement, 
including powers of determination over inclusion and exclusion 

for specific program activities. 

16. resource producing activities 

17. technical assistance for implementing innovations and changes 
Cooperative arrangements in education must provide the impetus, 

framework, and structure for development of new kinds of educational 
systems, incorporating new ideas, media, and operations. Cooperation 
is not simply another way of looking at shared services; it must reflect 
capabilities for the conceptualization and development of (1) new ways 
of conducting activities for the educational enterprise, (2) new ideas for 

generating programmatic systems for the educational enterprise, and (3) new 
support systems for education. 

The Unit is built on the basic assumption that the concept of local 
control of education is both a logical and a political reality; it attempts 
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to develop an innovative, generative, self-conceiving educational outlook 
within constraints imposed by the regular educational system. Such a Unit 
should build upon the given educational structure and serve to provide a 
catalyst for new ideas and developments; it should be synergistic. Unless 
the Unit develops the capabilities for generating innovative solutions and 
for developing innovative designs and systems in education, it does not 
represent something new in the educational structure; it becomes a structure 
for compounding present inadequacies. 

Thus, in developing long-range planning processes and procedures for 
evaluating the effectiveness of cooperative arrangements, the developmental 
steps must take into account underlying assumptions upon which the Unit 
is based and must reflect those assumptions both in planning stages and 
in evaluation procedures. A true cooperative moves through a number of 
stages from the idea of a single district ’’going it alone” in everything 
to the generation of new ideas, structures, systems and concepts for 
education. Figure 3 shows some of the developmental stages between single 
district operation and full cooperation. 

Clearly, in planning stages Unit personnel and governing groups must 
enumerate some basic and long-range goals to serve as guidelines. These 
goals must reflect alternative ideas and designs in education and must 
provide latitude for future development, field testing, and implementation 
of newer concepts applicable to education. 

The Unit must gain and retain its power through the participative 
nature of the organization; it should not be coercive, but should attract 
influence by provision of services not otherwise available to its clientele 
or membership. This in turn implies that individual units should have the 
option of ”buying. in” or of ’’buying out”, depending upon the perceived 
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Figure 3 . Flow chart indicating planning steps and progress toward 
in education, shown in developmental stages through time 






effectiveness of services rendered. "Holding power" of the Unit can then 
become a criterion variable for evaluation of program activities. Coercive 
membership, like tenure, presupposes incompetence and does not encourage 
organizational innovation. 

On the other hand, in order to initiate a cooperative Unit, it may 
be necessary for local units to pay a "membership" fee to generate 
resources necessary for the organization to become operational. After a 
specific trial period as determined by the governing board ( three years, 
perhaps) the Unit should have become strong enough to stand on its own, 
holding manbers through the quality and quanitity of services rendered. 

For the Unit to gain its initial impetus, some adjustments must be 
made at the outset by each agency cooperating under the jurisdiction of 
that Unit. Some of these adjustments might be as follows: 

1. relinquishment of some basic decision-making power 

2o agreement to participate as a component in a cooperative 
arrangement during initiation and field-testing of the Unit 

3. allocation of some resources for administration and development 
of the Unit 

4. contribution of personnel to work toward full implementation 
of the Unit 

5. cooperative attitudes toward restructuring control of some 
activities from local boards of the Unit’s governing board 

6. awareness that not all programs of the Unit will apply 
equally to all local districts within the Unit 

General Financial Considerations ; The Unit should be able to provide 
more and better services on a relatively more economical base than each of 
the structures which it was designed to serve. This in no way implies 
that the Unit v;ill cost less, or be cheaper than prior or present organiza- 
tions; the diversity of services and the potential for innovation, development 
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and self-improvement available through cooperation will be more economical 
than the same range and scope of activities would be if they were sponsor- 
ed by individual organizations or school districts. The strength of 
cooperation lies in its ability to generate new ideas, systems and pro- 
cedures for education in a relatively economical manner, not in reducing 
present costs for education. A regional cooperative Unit provides a way 
of making better investments in education, as well as more sound expendi- 

3 

tures for education. A regional cooperative Unit, due to its larger 
scope and unique position in the educational structure should engage in 
a few calculated "risk-taking” ventures. It certainly should allocate a 

percentage of its resources for "resource-producing” or "resource-creating” 

. . . 4 

activities; activities designed to provide future pay off in the educa- 
tional enterprise. Such activities might be teacher improvement projects, 
planning, research development, policy improvement, etc . Resource produc- 
ing activities are a form of investment. Whereas the single school dis- 
trict may be too small to set aside a large enough percentage of its 
operating budget for such ventures, the Unit should be able to provide 
assistance for its whole service area, especially in light of the similarity 
of problems among the districts in Region 1. Crucial elements, then, are 
the potential for: (i) growth and improvement, (2) economic efficiency, and 

^J. K. Galbraith, The Liberal Hour , New York: The New American Library 
(1964), p« 40. A Mentor paperback. The test of what a community should 
spend on a social service is what it can afford -- what it believes it can 
spare from other forms of consumption. The test of investment, by contrast, 
is what will pay for itself . ( emphasis added). 

, ^"Resource-creating” activities is a term borrowed from Policy Making 
for American Public Schools , developed by the National Academy of Education, 
March, 1969. The term is used here as in that work, pages 19-21. 
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(3) better educational opportunities for young people growing up in a 
dynaiTiic society. A cooperative Unit provides strength and credibility 
in the presentation of ideas to the legislature, the power structure, the 
general public, and the state education agency dtio to a larger constituency 
identifying and realizing similar problems in their educational endeavors. 

Goals : The determination and refinement of goals for a regional 

cooperative Unit is of primary importance especially early in its develop- 
mental stages. Goals should be based upon the criterion, ’*What can best 
be done cooperatively to improve education?” Since goals are desirable 
things to be attained, the basic development of goals should not be 
circumscribed by ” IvTiat is” or reality; the development of goals should 
enter the domain of ” What should be” or ”What ought to be”. Analysis of 
progress from what is to what should be can provide a focus for the evalua- 
tion process and should provide indicators of the efficiency and effective- 
ness of the Unit's activities. Some suggested general goals have been 
stated previously. 

Resource Allocation : Resource allocation decisions follow determi- 

nation of goals. Resource allocation includes not only general fiscal 
concerns, but in-kind contributions, space, and time contributions as well 
as overhead and maintenance. It is apparent that a percentage of the 
Unit's resources must go to organizational developmeiat and maintenance 
(internal). Expenses for organizational development should decline in 
terms of total percentage of allocation while percentage of resources 
"allocated for maintenance and program development ( external) should 
increase. As the service Unit reaches some point of equilibrium, the 
percentage of resources for maintenance should continue to decrease while 
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the percentage of resources for program development should increase. At 
some point a condition of "dynamic equilibriimi" should be reached where 
a predictable pattern of resource flow can be seen among the components 
( organization development, organization maintenance, program development, 
and program maintenance.) At all times, however, the Unit must retain 
part of its resources to allocate to innovative program development. 

These funds should be seen as "risk capital”. (See figure 4). 

In an analysis of resource allocation, time must be seen as an ex- 
tremely important resource. Time represents a closed system; i,.£. , there 
is a finite amount available and the decision to use a given amount of 
that time represents a commitment based upon a value orientation. Within 
the constraints of time, organizational goals must be evaluated so that 
there is a fair relationship between expected output and allocated input 
of time. 

The initiation of a new organization provides an opportunity to build 
in procedures for continuous evaluation and self-renewal. If these processes 
are installed iiriLtiallyy ithey will be recognized as a normal part of 
the organization and will not engender the resistance that would be forth- 
coming if they were to be imposed at a later date, at which time it might 
seem that evaluation were beiiig proposed to point out weaknesses in the 
organization. 

Program Operation : Planning activities should relate to projected 

program operation. Implementation of program activities will ultimately 
define the effectiveness of the planning activity of the Unit. If new 
systems for educatioh develop due to the synergistic and catalystic nature 
of the Unit, there will be a concomitant development of new roles and 
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responsibilities in education. 

A major function of a special service Unit should be to serve as 
a point of interface with other systems and to provide a locus and 
procedure for system integration. Since a regional cooperative. Unit is 
designed to serve a group of districts sharing coniraon problems, it should 
be able to synthesize their unique needs and then serve as an interface 
with the larger system in which the local education systems are operating. 

A cooperative Unit would operate between (1) the state education agency 
and the local systems, (2) any regional activities ( such as an educational 
laboratory) and the local systems, and (3) systems of higher education 
and local systems. A second aspect of the system integration process is 
the coordination function that the Unit can perform for individual school 
systems within the Unit's service area. Although primarily recipients of 
a coordination function, systems served by the special service Unit should 
have some agreement to share some of their decision-making responsibilities 
in terms of changes seen as good for the total service Unit area. 

In its rudimentary form, the Unit provides a framework for cooperative 
planning and problem-solving in education. Whereas the typical state 
education agency staff is too small to provide consultative service to all 
districts in most areas of need, the Unit can provide a new point of more 
intense contact with local districts and respond readily to regional needs. 

( Tlie same situation obtains in the relationship among the Unit, higher 
education and local schools). To insure the desired interactions, the 
Unit's controlling board must contain representation from the state educa- 
tion agency, as well as local schools, regional activities, and institutions 
of higher education within the Unit's geographic service area. 
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Governance : Governance for a regional cooperative Unit should be 

a function of a board broadly representative of local uuifs to be served 
and of cooperating agencies of the Unit. Specific governing relationships 
could be determined by the enabling or the mandating legislation. In 
terms of working relationships, however, the Unit's governing board shovild 
be made up of representatives chosen from the controlling boards of the 
Unit's clientele; professional guidance would come from committees or 
advisory groups. That is, the Unit's board should be structurally under 
the control of various local boards to be served by the Unit. (Figure 5 
presents a skeleton outline of governing relationships). 

Roles relative to operation : An organizational framework may 

encourage change and new things to happen, but people serving in new 
roles that have been rationally conceived to improve specific deficiencies 
are the factors that enable change to occur. The Unit can provide for 
program flexibility through employment of persons to serve a number of 
roles not usually identified as "normal education roles" and not usually 
provided in the traditional organization. This flexibility should 
stimulate program development and accelerate change processes. 

An example of a role that might be developed within the Unit is that 
of educational diagnostician, a person trained to identify specific learn- 
ing handicaps or impairments and who, in a team arrangement with a special' 
ist, should help to define individual programs of instruction for persons 
»ith learning disabilities or impairments. The diagnostician's role is 
that of a generalist in diagnosis. He is not an expert in special educa- 
tion or soffie clearly defined area of learning disability, but rather he 
Kas a general knowledge of problems which hinder normal learning in the 
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Figure 5. Skeleton diagram of goverance for the Unit, Note 
relationship vxhereby the Unit is controlled by 
its constituent members 



classroom. 



The diagnostician should make preliminary plans and referrals 



to experts. 

The Unit should provide access to specialists in newer instructional 
concepts, such as learning packages or individually prescribed instruction, 
or other programs designed to assist learning. Such specialists would 
not have as their major focus a subject area, such as science, but would 
be concerned with new approaches to learning. 

A regional cooperative Unit, although assumed to be a useful entity, 
must take initial steps to ( 1 ) define its role as an organization and (2) 
define new roles within the organization to be filled by personnel. These 
roles should be complimentary to those of local educational units and 
should be seen in the larger context of development and expansion of 
better educational services. Examples of such roles follow; some roles 
have been combined but may be expanded as demand for service increases. 

A brief summary of representative activities of each role is included. 

1. Educational evaluator . This office would maintain continuous monitor- 
ing and evaluation of new programs of the intermediate Unit and would 
have responsibility for specific evaluation tasks as designated by 
local districts participating in the Unit structure. The evaluator 
would have inservice responsibilities for working with educational 
staff in the evaluation process, especially in developing ways of 
monitoring student achievf“aent. 

2. Inservice director . It would be the responsibility of the inservice 
director continuously to refine and develop new inservice approaches 
aimed at continuous upgrading of educational personnel. T).iis person 
probably should have; a joint appointment in a college of ediicatlon 
with responsibilities for teacher education and/or student laboratory 
or field experiences, 

3. Media and Comtaunications System specialist , (self-explanatory) 

4. Materials development and clerical assistatice. (self-explanatory) 



5. Program developer and information officer . This person would have 
major responsibilities for helping in proposal development and in 
providing information for local schools and/or local district use, 
as well as preparing brochures or other public information materials, 
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6. Federal and state program coordinator . This person would have major 
responsibility for maintaining files and current information on new 
federal and state programs and mandates. Clerical assistance in this 
office would be available to process paperwor?ic and forms for all 
federal and state programs in the service area. This office would 
work closely with the program developer to be sure that proposals 
and grants were developed in line with the intent of legislation 

and guidelines. 

7 . Personnel coordinator . Ihis office would be responsible for maintain- 
ing a personnel file for the entire area served by the Unit and, act- 
ing with the direction and assistance of local personnel, would engage 
in recruitment activities based upon a personnel plan developed for 
the service area of the Unit, as well as for the staffing of the Unit 
itself. 

8. Research and special programs . Research activities of the Unit should 
be limited, but some research will be necessary in the development of 
new programs and evaluation plans. Major functions of this office, 
however, would be to serve in coordinating experimental or special 
programs which should be operated or directed through the Unit, to 
field-test new programs, to help identify needs in the service area, 
and to coordinate research activities of higher education institutions 
in an attempt to find applicable solutions to local problems. If new 
programs prove successful, they should be implemented in local schools. 
This office would then serve as a focal point for insertion of newly- 
tested programs, and would be seen as a facility for developmental 
activity. This office would be, in effect, a gadfly to introduce new 
ideas throughout the intermediate Unit service area. 

9. Personnel for data processing activities . Total data processing 
activities for the service area of the Unit should be a central 
element in the development of the Cooperative; they will provide 
the first-line of programs for the Unit and provide impetus for 
all new program development. Initially this office would be 
concerned with computer assisted administration ; as services 
develop and grow, computer assistance 'Jill be ayailable for 
specialized programs of instruction. It is important that 
initial planning of data processing activities includes cap- • 
abilities for data compatibility, planning activity, space 
accounting for identification of classrooms or buildings that 
could be used for special programs, processes and procedures 

for updating baseline data, as well as other activities generally 
seen as related to data processing. (Refer to the final report. 
Feasibility Study for a Regional Data Processing Center in Fayette , 
Greene , and Washington Counties in Pennsylvania for the Regional 
Planning Project . Title III ESEA . Systems Development Corporation, 
April, 1969, for more comprehensive presentation of the functions 
of this office.) 

10. Educ at ion and community planner , (self-explanatory) . 

- 20 - 



22 



11. special pro ject office . This office should have one or more 
people available to set up and operate some special projects. 

One or more of these positions might be reserved for intern- 
ships from within the local systems or from higher education. 

Each of these roles represents possibilities for change, growth and 
development. Some of the roles will be filled as needed early in the 
growth of the Unit; others will provide guidelines for growth. Some 
of the roles represent the core or backbone for the ve..y initiation of 
the Unit. Initial planning and goals will identify those roles which 
will be considered necessary for the early stages of the Unit. Develop- 
mental roles will emerge after the Unit has been operating for awhile. 

Summary 

The Unit should serve as a source for the planning, development, 
initiation and evaluation of educational programs and change. Cooperative 
action controlled within the local districts of the Unit will provide a 
broader base for educational development and improvement. This paper is 
an attempt to provide some structure, framework and guidelines to assist 
in the planning and development of an intermediate unit to provide regional 
education services. 

One objective of cooperative planning in providing regional education 
services is to encourage the development of careful evaluation of organiza- 
tional activities in such a way that provision is made for the addition of 
programs that have been developed and tested and for the deletion of programs 
which the new programs are designed to replace. A philosophy regarding 
developmental activity must be based upon the planning model. An intermedi- 
ate unit for cooperation in providing regional education services probably 
has a unique function in development as it relates to local school districts. 
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Provision for Evaluation 



Introduction 

Evaluation of a regional cooperative should be designed to measure 
both quantitative and qualitative change. Based on the assumption that 
educational cooperative endeavors should be designed to encourage new 
systems, ideas and procedures in education, quantitative measures will be 
more helpful than qualitative measures in assessing the Unit's general 
effectiveness in accomplishing stated objectives. Evaluation is simpler 
the more clearly are stated the objectives. However, objectives relating 
both to qualitative and to quantitative elements should be constructed. 

Since a prototype has a higher per unit cost than an oh-going activity, 
initial evaluation of change and of new or demonstration activities should 
focus primarily on quantitative ( e. , effectiveness) aspects. The 
connotation of failure must be judiciously avoided in the evaluation process, 
since failure implies an end product rather than a process and procedure for 
developing new or better approaches to the attainment of goals. Evaluation 



must be seen as supplying a feedback link to encourage generative action 
resulting in growth and self -renewal. 

There are at least two broad frames of reference for evaluation 



efficiency and effectiveness, and the evaluation role of the Unit serving 
as an external evaluator for programs of local districts. One role 
hypothesized for the Unit was that of evaluator. Thus, districts should 
be free to call upon the Unit to assist in designing, developing and con- 
ducting evaluations of local activities. On the other hand, internal 
evaluation procedures both of the Unit's organizational operation 



activity of the Unit: evaluation of the Unit’s operation and of its 
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-- maintenance and development -- and of the programs developed and operated 
by the Unit are necessary. This section will deal in general ways with 
procedures for establishing guidelines and a model for evaluation. 

Numerous evaluation models are appearing on the educational scene. 
Probably many of these could be adapted to an evaluation process for the 
regional Unit. On the other hand, a synthesis of the better points of 
several models should provide a more workable framework for evaluation. 

It is also probable that several evaluative techniques will be useful due 
to the complexity and comprehensiveness of the regional cooperative Unit’s 
proposed programs. Care must be taken at the outset to determine, as well 
as possible, kinds of data necessary for evaluation to insure comparability 
of data at a later time. Immediate evaluations may be guilty of collecting 
moru data or more detailed data than necessary. This sin, however, is 
probably more forgiveable than not collecting enough. Data once collected, 
may not be used but it is difficult to generate data once the collection 
stage has passed. 

Evaluation should provide signposts or guideposts for continuity, 
change, and innovation, as well as an assessment of the status quo . 
Evaluation should be seen as a process and not as an end product; a process 
viewed through time. All too of tett' evaluation is seen as a "final report" 
on an activity or on an organization; worse yet, as a fine’’, report to be 
shelved and forgotten. Evaluation must be recognized as continuous feed- 
back providing direction for the attainment of objectives or goals; for 
the development of new goals, directions, and programs; and as a mechanism 
to instigate continuous self -renewal of the organization. 

Evaluation should be planiisd carefully within a logical and flexible 
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framework. A sound and workable evaluation model depends upon the clear 
statement at the outset of the program objectives in a variety of behavioral 
steps. Secondly, global program objectives seem necessary. Each global 
program objective should be capable of being divided into levels of speci- 
ficity which lend themselves to a reasonably accurate analysis of their 
probable achievements. 

The evaluation model must take into account the factor of time. 

Progr^s need time in order to prove their worth and effectiveness. Time 
must be a factor in the evaluation model to allow for the accomplishments, 
or the partial accomplishment of various goals as stated in the planning 
phase. Evaluation implies that something is either going somewhere or not-- 
that change is taking place. This is especially true if time is included as 
an element, suggesting that evaluation will identify whether or not some 
change has occurred over time. Some attention should be paid at the outset 
in defining and understanding some key concepts such as change and innovation, 

since they will be the focus of the evaluative activity. ' 

> 

Change and Innovation 

Some research on change has suggested that a system's ability to 
change is directly relational to its opportunity to provide for observation 
of newer procedures, ideas, structures and methods. The regional- cooperative 
Unit must incorporate built-in procedures encouraging or providing opportuni- 
ties for demonstration, dissemination, diffusion, and adaptation. 

Change can be defined as a process aimed at a goal. Change is a 
j jrocess , not a thing; the end product of change may be a thing, or it may 
lae a new process. Based upon such assumptions, project evaluation must 

- 25 - 



O 

ERIC 



A 



26 



I 

I 

I 

I 

I 

J 

I 

I 

I 

I 

I 

I 

1 

I 

I 

I 

I 




provide feedback for continual change and readjustment of program direction, 
rather than simply be an exercise for a final report to be filed. Evaluation 
aimed at the development of a final report is, in effect, evaluation of an 
end product or a thing; continuous evaluation as feedback for decision- 
making is evaluation of process. 

The Definitional Problem ; One thing is apparent in the definitions of 
innovations and change which appear in Appendix A of this paper; innovation 
is not necessarily seen as invention ( _i.£. , something that is unique or 
entirely new), but rather that innovation may be relative to a locality. 
Innovation may be the recombination of knovm elements into new programs or 
the adaptation by local districts of a proven program to meet a local need. 
Recommendation XI of the Subcommittee on Education of the Committee on 
Labor and Public Welfare of the United States Senate posits the following 
definition for innovation. 

XI. Educational innovation is a. new or different concept, 

methodology, organization, or program that is systematically 
introduced into the classroom, school system, and/or the state 
as a whole. ^ 

Thus, an innovation can be something adapted to locdl needs through 
a recombining of proven educational activities into a new model designed 
to alleviate needs identified at the local level. The innovation, when 
demonstrated and diffused, should have promise of alleviating needs in 
other locales with adaptation of specific elements to the local needs. The 
regional cooperative Unit is such an idea or concept. If evaluation of 

^United States Senate,, Subcommittee, on. Educatioa of the Committee on 
Labor, and. Pub lie. Welfare, Notes and. Working Papers Concerning the Adminis - 
tration of Programs . Authorized .Under. Title . Ill of. Public Law. 8S -.10. , The 
Elementary and Secondary Education Act of 1965 as Amended by Public Law 
89-750 , April, 1967, p. 37. 
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activities of the Unit is to relate to change, some notion of change and 
innovation must be clearly held to provide a basis for that evaluation. 

Stufflebeam, in the CIPP evaluation model describes four kinds of 
evaluation: context evaluation, input evaluation, process evaluation, and 

product evaluation, ' OQLyv:oae o£! these elements , product evaluation, focuses 

f 

on an end product; the other three elements are primarily concerned with 
process. The emphasis on proceaa.may suggest that the classical experiment 
tally designed and controlled evaluation procedures are not the best, ' 
especially for global measures of change. 

Historically , educators have attempted to establish experimental 
models and designs as a basis for evaluation. This procedure is parti- 
cularly well suited to product evaluation, but less appropriate to process 
evaluation. Product evaluation techniques may not be applicable to evalua- 
tion of change in the true sense. Experimental design as a procedure for 
evaluation implies that rigid controls are maintained; no alterations are 
made in any central element in the method, technique, or content being 
evaluated lest the modification render an invalid evaluation. This hinders 
making adjustments in the program to reflect changing needs based on 
preliminary feedback. 

Cuba makes the following distinctions between product and process 
evaluation, r . 

Generally speaking, the traditional rules of experimental 
design and field control are rigorously, invoked. The essential 
task of traditional evaluation is to judge . 

Modern concepts of evaluation veer sharply from these 
traditional ideas. Evaluation is seen less now as a judgemental 
device than it is as a decision-making device; the purpose of 
evaluation is to assist in the making of decisions. Since there 
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are many decisions to be made, there are also many 

kinds of evaluation, some of which are sharply different 

from the product evaluation described above.® 

Thus, It appears that the most Important kind of evaluation for the on> 
going Unit structure Is process evaluation. Process evaluation should be used 
with on-going educational programs of the Unit and product evaluation 
should be used after the conclusion of a specific program , before It 
Is adapted or adopted ^ toto In local schools within the Unit's service 
area. 

Evaluation procedures may take many forms. If basic objectives of 
the Unit are to Initiate change and provide better and newer educational 
services, then some evaluation of the effectiveness of the Unit will 
Include a summary of whether or not basic objectives are met; , 
whether or not change Is taking place In a planned manner. Gross measures 
of the following kinds of things can provide Indicators of the extent to 
which a program of the Unit Is meeting basic objectives. 

1. Expanded opportunities for students. 

2. Expanded experiences for students. 

3. Expanded numbers of choices for students. 

4. Increased flexibility within choices available. 

5. Economic feasibility of program. 

6. Increased program activities. 

7. Better administration support systems. 

8. Numbers of new programs available. 



®Ibld. . pp. 306-307. 



Evaluation Design 



The evaluation design described in this model is essentially an 
extension of Stufflebeam's CIPP evaluation model. Elements incorporated 
in the extension of the model Include some taken from the EPIC model and 
some synthesis taken from the evaluation model for Title III programs as 
developed in the state of Tennessee.^ As such, the evaluation model or 
design includes four separate phases: Phase I, context, status or assess- 

ment evaluation; Phase II, input or planning evaluation. Phase III, process 
or operational evaluation; and Phase IV product, or final evaluation. 

Within each of these evaluation stages can be developed procedural 
outlines such as those shown in figures 6 and 7.( pages 32 and 33 ). 

Although there will be modlflOatlons in the procedural guidelines for each i> 
phase of evaluation, the general model for evaluation procedures should be 
applicable. 

Each of the steps of the evaluation design are logical extensions of 
planning activities, and should compliment planning activities of the Unit. 
While planning may be seen as preparation for making decisions and providing 
the framework for action, evaluation can be seen as a review of decisions 
in light of certain data, criteria, or some changes that can be documented, 
and as a procedure for exploring the need for new decisions. Contl'^uous 
operation of an organization of program is seen as an administration or 
management function and is related to both planning and evaluation. 

There now follows a summary of four phases or klt^s of evaluation. 

The phases should be seen as continuous; that is, although they are divided 
into phases for discussion they are really all parts of a general evaluation 

^Appropriate references are shown in the bibliography. 
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process. Implementation of each phase will depend upon planning, develop* 
ment of objectives, and the following of a logical process. In some cases 
there Is a discussion of what should be done In each phase and, as applicable, 
a discussion of what has already taken place in Region 1 of Pennsylvania. 

Phase I evaluation ( context, status or assessment): Prior to Phase I 

evaluation there Is an Initial planning stage which leads to decisions to 
do something. These initial planning decisions should lead directly to the 

application of Phase I evaluation. In this phase, a general assessment of 

vV 

need Is conducted. This assessment will also provide some baseline data, 
as well as guidelines for future action. In some cases, sampling procedures 
and analyses are applicable; In other cases population data and compilation 
of averages and ainple comparisons are applicable. In Region 1, initial 
status or assessment results were Incorporated In a planning grant proposal 
and subsequently In the definition of needs and In the development of plan- 
ning documents for each district. After initial assessment .and Identfflca^io::. 
tion of need, activity is initiated under Phase II evaluation. Some adjust- 
ment and expansion of needs from the initial planning stages will be made 
as a result of operational procedures of the context or assessment phase. 

Phase II evaluation ( Input or planning) t ic ies i i iU y tfdawctly from 

activities conducted under Phase I evaluation. ( Results of Hiase I 
evaluation led. In Region 1, to. development and imp lonenta tion of planning 
activity supported by a planning grant.) Some Phase II evaluation activi- 
ties have been conducted under the planning grant and are incorporated in 
the operational plan as goals, objectives,, strategies, procedures, and 
various other Judgmental criteria. Also, the legislative mandate and 

requirements of the Department of Education provide input or planning 
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data for the Ratification of the next steps of development. 

Additional planning or Input evaluation steps must be taken after 
operation has begun to provide a basis for specific planning relative to 
implementation of discrete program elements of the Unit. In other vords, 

each activity of the. Unit should follow through basically the same four 

/ 

phases of evaluation as did the development of the Unit Itself. A tentative 
deslgh for guiding Input or planning evaluation Is suggested In Figure 6 
on the next page. 

Figure 6 relates specifically to an evaluation of majjor organizational 
objectives and to determining the feasibility of new programs thrusts for 
the Unit ( a form of planning); Figure 7 on page 33 relates to analysis of 
new programs to implement organizational goals or major program thrusts. 

Both formats are similar, however, as they outline logical steps In 
implementing an evaluation activity. Questions of technique -- sampling, 
comparisons, analyses -- are <nnltted from the general design. 

Kiase III evaluation ( operational or process evaluation); Hiase III 
will commence after the Unit and/or Its programs have operated for a given 
period of time. Procedures for conducting process evaluation mu?*^ be 
established beforehand to serve as guidelines for operation. However, some 
recycling of operational evaluation processes may be n^^cessary as data are 
fed back to the operational scheme and as monitoring provides a decision- 
making base. This may be true since process evaluation should provide 
periodic checkpo. its to serve as bases for changes In decisions. Operational 
or process evaluation Is not a *'clean*’ experimental activity since complexity 
of the operational evaluation scheme Is circumscribed by the Inability to 
control all variables experimentally and the Inappropriateness of random 
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Figure 7 . Major Stages in Design for Developing and Evaluating Innovative 
Instructional Systems (Figures 6 and 7 were adapted from a 
model described by Dr. Jacob Shapiro). 
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selection or assignment. 

Before any evaluation of change may be undertaken, there must be a 
lapse of time over which some change may be identified. Operational 
evaluation cannot be expected to produce results until the project has been 
operating long enough to effect some change. Development of means, analysis 
of participant evaluation, and analysis of gross measures of increase or 
decrease indicative of change may provide systematic bases for decision 
making and the rudiments of procedures for self-correction and self-renewal. 
Subjective participant evaluation on the part of pupils, the community, and 
educational personnel may be valuable in process evaluation. Growth of use 
of Unit services and interest on the part of local districts for increasing 
participation in the Unit and for increasing the scope of Unit activities 
will provide indicators of success. Ultimate evaluation measures for the 
Unit may be predicated on changes in students; initial evaluations may be 
based upon quantitative measures or gross program elements. Wherever 
possible, evaluation should be based upon behavior changes. 

Phase IV evaluation ( product or final): Phase IV will be a summation 

of process or operational evaluations, resulting in some value being put 
on the end product. If applicable, statistical analysis pertinent for 
final evaluation techniques will be used. These techniques will be 
employed wherever appropriate field controls, random assignment or 
selection, and order of data ( ordinal, nominal, etc .) are available. 

Final evaluatiom procedures Mteuld be to 'dfeteflaklne- the' effec 
of specific programs that have been developed and pilot- tested under con- 
trolled circtimstances. Such product evaluation must also show econcnnic 
considerations ( accountability) as well as program improvements or lack of 
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improvement. Prior to adaptation of any specific program by a member 
district of the Unit, the program should have been subjected to careful 
and controlled product evaluation. 

Insertion of elements from Unit programs in schools will provide 
quantitative data on the progress of the change process from awareness to 
diffusion and finally institutionalization of innovations within an operat- 
ing system. The rate of institutionalization of program elements of the 
Unit will also demonstrate the efficacy of cooperative action between and 
among school districts with similar educational problems, in that activities 
which have been field-tested in similar settings can be adapted readily to 
new districts. 

Evaluation of these measures will be by relatively simple procedures 
such as evidence of continuing participant interest as reflected in 
implementation of ideas from the Unit in more classroom settings in a 
single school, school district, or throughout the region. 

Summary 

A. concept and design for evaluation has been outlined in this section 
of the paper. A summary of scheme for the procedure has been set forth in 
figure 8. This figure has been adapted from page 14 of the Design for 
Tennessee Assessment and Evaluation of Title III , ESEA . Fi^re .8 shows 
each of the four primary phases of the projected evaluation procedure 
and their relationships to each other. Techniques of evaluation within 
each of the phases will vary. The basic models from figures 6 and 7, 
pages 32 and 33 should help provide direction for the evaluative process 
within the phases. 
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Initial Planning Activity 




Figure 8 . . Evaluation Djeslgn.Proceas Diagram adapted from Design for 
Tennessee Assessment and Evaluation of Title III . ESEA ; A 
report developed by Dr. F. K. Bellott at Memphis State 
University for the State of Tennessee, Department of Educa 
tion, J. H. Warf, Commissioner, 1969. 
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Planning and evaluation are seen as continuous elements of a total 
management process. Planning established the values and the philosophical 
points of decision as well as the direction toward which the organization 
should move. Whereas, the role of planning is probably to give preparation 
for making decisions ( direction, values, philosophy, etc . ) , the use of 

I ' 

evaluation is to review decisions in light of some criteria established 
to determine whether the objectives are being met or whether new decisions 
need to be made. 

When a new organization is established, it is imperative that the 
evaluative function be iiaterwoven so that the organization sees evaluation 
not as a threat, but as a process for continuous self -renewal; the new 
organization should program an evaluation procedure into itself at the out- 
set so that the staff of the organization can feel comfortable with a built 
in process of self-renewal . Evaluation should provide a feedback to the 
organization relative to the effectiveness of the organization's activities 
As new programs are tried, tested, and added to on-going activity, old 
programs should be replaced. The concept of continuous evaluation should 
remove most threatening elements from the addition and replacement process. 
This will, however, only be possible if personnel do not see a ’Vested 
interest” in every program, but rather see possibilities for continued 
change, growth and development as the most critical elements in the 
organization. 

Thus, evaluation is an essential concomitant of the planning process. 
Both planning and evaluation will be esseatial elements for the development 
of any plan for cooperation in providing regional education services. 
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APPENDIX A 

DEFINITIONS OF INNOVATIONS 



The following definitions of innovation may provide some basis 
for the development of the Region 1 Intermediate Unit and some guidelines 
upon which to base subsequent evaluations of the effectiveness of the Unit 
in generating change and innovation. Characteristics of all of the defini~ 
tions are that innovation Ais not necessarily invention, and that innovation 
is relative to the locale. These definitions- are particularly applicable 
if a specific intermediate unit is designed to serve a particular area 
and its needs, rather than to provide uniform but general services of 
similar nature as dictated by some central source. • 

DEFINING INNOVATION ^ / 

J. 

Several special consultants developed definitions of 
i innovations. Six of 'them are included here: 

Everett Rogers defines an innovation as-- 
an idea perceived as new by an individual. This is a sub- 
;; * jective definition of innovation; it is based on the 

c assumption that what matters is whether the idea is new 

to the receiver, rather than whether it is objectively 
new in the eyes of experts. If we seek to understand 
the individual’s behavior in regard to innovation, we 
must view the situation and the idea from the point of 
view of the receiver.^' 



Harry Fassow defines it in this manner: 

To the extent that these school systems, alone or in 
consortium with other systems, are tackling significant 
valid theory and research to develop programs and pro- 
cedures which are undoubtedly 'new' for that system — 
such school systems are gar tic ipa ting in a kind of 
educational innovation.^” 

47 Rogers, p. 146. 

48 Passow, p. 221. 
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Glen Heathers writes that-- 



a proposal may be innovative in relation to a particular 
innovation, or through making novel contributions to the 
design and implem,:ntation of local: school improvement 
programs. A third possibility is that a project may be 
innovative only in the limited sense of introducing cer- 
rogram features within the locality for the first 



James Finn comments that -- 

nothing is likely to be, strictly speaking, innovative in 
the Kuhn sense of creating a paradigm never before set: 
forth * * * Educational innovation * * * cannot be class- 
ified either as something completely new or something that 
is only new within the locality or region. Rather it should 
be thought of as existing on a continuum. 50 

Harold Gores uses this categorization; 

(1) ’'Catch-up" Projects --where the need is for the basic 
tools of education. They represent poverty in education 

and are an obligation to be met. 

(2) "Innovative Projects"--T?here the quest is to import 

a practice successfully demonstrated elsewhere. This category 
constitutes the heaviest obligation and puts a special burden 
on the dissemination of information so that the schools will 
know where discoveries have been made. 

(3) "Inventive" Projects — where the search is for a new 
answer to an old question, or a workable answer to a new 
question. 51 

Hilda Taba also believes that in order to deal with the 
question of innovativeness, one needs to consider innovations 
according to differences in levels and types: 

First, there are innovations that create an entirely new . 
perspective on curriculum, teaching, learning, or instructional 
materials and media. These innovations require reconceptualization 
of content, teaching strategies and learning processes. 



49 Heathers, p, 184 

50 Finn, p. 331 

51 Gores, p. 294. 




A second type of innovations represent Implementing of 
already created models, testing them and adapting them to 
local conditions, such as trying out a new mathematics 
curriculum or incorporating a method of discovery learning. 

A third type of innovation is that of altering the admin- 
istrative arrangements, such as installing team teaching, 
changing the manner of scheduling, or introducing new 
media. These types of innovations are of little worth unless 
they are accompanied by appropriate changes. 

A fourth type of innovation is extending practices which 
are not in themselves innovative into areas that have lacked 
them. This is innovation in a local sense only. 52 

If goals of the Unit are conceived as two-fqld — improved 
organizational activi^.y (maintenance) and the generation of 
change and innovation leading to new and better ways of 
conducting education — then some basis for evaluating the 
Unit's effectiveness in terms of change must be defined. 

Further, there must be some generally accepted view of what 
change and innovation are. Perhaps some of the definitions 
in this appendix will be of assistance. 

4 



S^Taba, p. 117. 
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